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ABSTRACT 

The  degree  of  uncertainty  in  a  track’s  position  provides  an  indication  of  how  best  to  allocate  sensor  resources.  In 
this  paper,  we  discuss  the  use  of  this  track  parameter  for  two  tasking  activities.  First,  we  consider  determining 
resolution  selection  for  electro-optical  (EO)  sensor  tasks.  Secondly,  we  integrate  our  knowledge  of  kinematics  into 
the  process  of  information-theoretic  task  selection.  We  consider  the  use  of  optimized  selection  of  the  resolution 
for  imaging  sensors  such  that  we  maximize  what  we  term  the  probability  of  acquiring  a  target.  The  probability 
of  acquiring,  if  a  particular  resolution  is  chosen,  is  a  function  of  both  the  probability  of  the  target  being  within 
the  sensor  footprint  as  well  as  the  probability  the  target  is  detected  given  it  is  in  the  footprint.  In  the  process 
of  selection  of  sensor  tasks,  Toyon  employs  an  information-theoretic  metric.  We  apply  conditionals  on  the 
entropy  that  are  a  function  of  the  uncertainty  with  which  the  track  will  actually  be  detected  during  the  sensor 
task.  A  tracker,  based  on  the  Kalman  filter,  is  used  to  provide  an  estimate  of  position  and  an  associated  two 
dimensional  error  covariance.  The  kinematic  information  is  used  to  compute  the  probability  a  target  is  within 
a  footprint,  whose  size  is  based  on  the  resolution.  For  simulation  we  employ  the  high  fidelity  Toyon-developed 
SLAMEM™  testbed. 
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1.  INTRODUCTION 

Automated  sensor  management  continues  to  draw  increasing  levels  of  interest  through  both  research  and  system 
development.  While  previously  it  may  have  been  possible  for  a  pilot  or  aircraft  operator  to  task  and  collect  data 
from  on-board  sensors,  the  expanding  number  of  sensors  and  the  raw  amount  of  data  make  this  task  daunting  for 
even  the  most  capable  individuals.  Thus,  the  operator  is  often  relieved  to  relinquish  this  control  in  order  to  obtain 
higher  level  knowledge,  such  as  threat  warnings  or  automated  search  of  a  region  of  interest.  However,  the  problem 
of  sensor  task  creation  and  ordering  its  execution  is  complex,  with  many  techniques  available.1  Figure  1  shows  a 
high-level  overview  of  a  typical  Intelligence,  Surveillance,  and  Reconnaissance  (ISR)  architecture.  Here,  multiple 
stand-off  assets,  whether  manned  or  unmanned,  possess  one  or  more  sensors  that  can  be  tasked  with  different 
parameters,  such  as  resolution  or  direction.  In  this  paper,  we  discuss  the  role  uncertainty  in  a  target’s  location 
can  play  in  both  sensor  task  creation  as  well  as  schedule  creation.  In  particular,  we  consider  electro-optical 
(EO)  sensors  with  variable  resolutions  and  propose  a  method  whereby  an  optimum  resolution  can  be  selected. 
Secondly,  we  consider  an  enhancement  to  Toyon’s  information-theoretic  approach  to  schedule  evaluation  in  which 
the  certainty  of  a  target’s  location  is  integrated.  In  addition  to  analysis,  we  present  simulations  that  indicate 
possible  gains  in  performance. 


2.  EO  TASK  CREATION 

Certain  imaging  sensors  have  the  ability  to  vary  their  resolution.  Thus,  aboard  many  unmanned  aerial  vehicles 
(UAVs)  the  operating  resolution  can  be  selected  during  task  creation.2  Typically,  an  operator  will  initially 
choose  a  large  image  footprint  and  then  zoom  in  when  a  target  of  interest  is  discovered.  Conceptually,  we  seek 
to  provide  a  similar  automated  approach  based  on  some  metric  of  performance  that  decides  what  an  optimum 
resolution  would  be  for  imaging  a  particular  track.  The  role  of  EO  task  creation,  and  the  associated  algorithm 
for  performing  dynamic  resolution  selection  (DRS)  is  shown  in  Figure  2.  The  process  consists  of  looping  through 
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Figure  1.  ISR  Architecture. 


all  possible  tracks,  determining  whether  an  EO  sensor  can  be  used,  and  then  finally  selecting  the  resolution  that 
would  yield  the  best  performance  for  that  track. 

Intuitively,  there  will  be  an  optimal  resolution  such  that  the  probability  of  obtaining  sensor  information  on 
the  target  is  maximized.  We  will  refer  to  this  event  as  acquiring  the  target.  Thus,  when  a  sensor  image  is 
placed,  the  two  possibilities  {^4,  ^4}  denote  the  set  whereby  the  target  is  either  acquired  or  not.  The  probability 
of  acquiring  the  target  can  be  written  as 

P(A)  =  P(A\F)P(F)  +  P(A\F)P(F)  (1) 


where  F  and  F  denote  whether  or  not  the  target  is  located  within  the  sensor  footprint.  The  second  term 
P(A\F)  =  0  is  due  to  the  fact  that  if  the  target  is  not  located  within  the  sensor  footprint  then  it  cannot  be 
acquired.  The  term  P(A\F)  will  increase  as  the  resolution  becomes  finer.  This  is  due  to  the  fact  that  with  finer 
resolution,  it  is  more  likely  the  sensor  can  discern  the  target  from  other  objects  and  ground  clutter.  The  second 
term,  P(F),  is  inversely  proportional  to  the  fineness  of  the  resolution;  with  finer  resolution  the  sensor  footprint 
size  will  decrease  making  it  less  likely  the  target  is  within  the  footprint.  We  see  the  two  terms  that  generate  P(A) 
are  inversely  dependent  on  resolution.  Intuitively,  we  expect  there  to  be  a  particular  resolution  that  maximizes 
P(A). 

To  simplify  notation,  we  let  Pjj  =  P(A\F).  The  term  Pn  denotes  the  probability  the  sensor  is  able  to  detect  the 
target  given  that  it  is  within  the  sensor  footprint.  These  terms  are  generated  by  Johnson’s  criteria.3  According 
to  this  criteria,  the  probability  of  detection  (Pd)  can  be  computed  with  the  equation 
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where  E  =  2.7  +  .7 r  and  r  is  computed  as  r  =  N/N^q.  For  this  equation,  N  is  the  number  of  resolvable  cycles 
across  the  target  and  N^o  is  the  line  pair  criteria,  which  for  our  purposes  will  be  detection  performance.  The 
IV50  values  were  derived  from  field  tests.  We  compute  Pd  as  a  function  of  resolution  and  target  type.  Because 
there  are  different  probabilities  associated  with  a  track  being  of  particular  target  types,  and  associated  Pd s,  we 
choose  the  Pd  associated  with  the  most  likely  target  type. 

Additionally,  we  define  the  expectation  that  the  target  will  be  in  the  footprint  as  Pj  =  P(F)  .  This  value  can 
be  computed  by  integrating  the  probability  density,  corresponding  to  the  target  position,  over  the  region  inside 
the  sensor  footprint.  Unfortunately,  integrating  in  this  manner  will  not  lead  to  a  solution  that  is  straightforward 
to  compute.  Thus,  we  seek  an  approximation  that  yields  a  more  manageable  form.  Our  method  is  depicted  in 
Figure  3  and  discussed  in  the  next  section. 
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Figure  2.  Location  of  DRS  within  sensor  tasking  architecture. 


2.1.  Finding  an  Approximation  for  Pj 

Here,  we  provide  a  method  for  determining  the  probability  the  TOI  lies  within  the  EO  spot.  Because  we  have  a 
model  for  the  position  uncertainty,  Pj  can  be  found  by  simply  integrating  over  the  probability  density  function 
corresponding  to  the  position  of  the  target.  Our  model  is  a  joint  Gaussian  density  given  by 


,xAx'v)  =  { (^f-)  -2* 

where  px  and  p,y  are  the  means  of  the  x  and  y  target  positions.4 
variances  and  p  is  the  cross-correlation.4 
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Similarly,  <r2x  and  cry  are  the  associated 


Figure  3.  EO  footprint  and  position  error  covariance:  (a)  ground  coordinates  (b)  transformed  coordinates. 


In  order  to  decouple  the  coordinate  positions,  thereby  easing  integration,  we  use  a  transformation  matrix. 
This  begins  by  obtaining  an  eigendecomposition  of  the  covariance  matrix  which  is  given  by  E  =  UDUT.  Here, 


the  matrix  U  =  [U1U2]  holds  the  eigenvectors  and  the  diagonal  matrix  D  = 
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eigenvalues.  The  resulting  transformation  matrix  C  =  ^U\/lD^  rotates  and  scales  the  covariance  matrix  of 
our  joint  Gaussian  such  that  it  becomes  an  identity  matrix.  An  example  EO  spot  once  the  transformation  matrix 
C  has  been  applied  is  shown  in  Figure  3. 

Once  the  new  coordinate  system  has  been  constructed,  we  find  an  approximation  of  the  density  over  the 
resulting  footprint.  In  our  new  coordinate  space,  we  note  the  axis  variables  are  now  given  by 
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with  the  resulting  probability  density 


fx',Y'{x',y’)  =  —exp 
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As  can  be  seen  in  Figure  3,  our  approximate  area  consists  of  a  rectangle  formed  by  the  longer  side  of  the 
transformed  spot  along  with  a  line  perpendicular  to  this  longer  side,  centered  midway  through  the  short  side. 
Noting  that  the  resultant  probability  density  is  isotropic,  we  can  integrate  over  the  EO  spot  along  the  u  and  v 
axes  shown  in  Figure  3.  That  is,  for  the  approximate  EO  spot  area  Z  we  have 
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where  L  and  S  are  the  side  lengths  for  the  approximate  EO  spot  and  erf  is  the  error  function.4 
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Figure  4.  Probabilities  as  a  function  of  resolution. 

2.2.  Searching  for  Optimal  Resolution 

The  process  of  choosing  the  best  resolution  requires  searching  through  the  possible  values  and  computing  Pa 
for  each.  Thus,  the  resolution  that  maximizes  Pa  is  selected  for  the  candidate  task.  Here,  we  consider  an  EO 


sensor  that  has  a  finite  set  of  resolutions,  denoted  by  ground  sample  distance  (GSD).  In  this  case,  the  algorithm 
can  simply  select  the  GSD  that  maximizes  the  probability  of  acquiring  the  target.  To  lend  an  intuitive  feel 
for  the  tradeoff  between  Pd  and  Pj  which  provide  the  resulting  value  for  Pa,  example  values  are  plotted  for 
a  range  of  GSDs  in  Figure  4.  For  this  example,  Pd  is  computed  with  r  =  1/GSD  and  the  footprint  width  is 
L  =  S  =  3/ GSD.  As  can  be  seen  from  the  figure,  there  is  a  maximum  value  for  Pa  over  the  range  of  GSD  values. 

3.  USING  TRACK  LOSS  PROBABILITY  IN  TASK  SELECTION 

Toyon  employs  an  information-theoretic  approach  to  sensor  schedule  evaluation.5,6  Thus,  we  seek  to  reduce  the 
total  level  of  uncertainty  in  track  identification.7  Using  Shannon’s  information  theory,  entropy  for  the  random 
variable,  track  identification,  is  given  by 
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where  pn  is  the  probability  that  the  target  belongs  to  the  nth  of  N  target  types.8  Information  theory  for  sensor 
tasking  relies  on  the  ability  to  compute  expected  entropy.  We  then  select  the  sensor  schedule  that  minimizes 
the  total  expected  entropy  for  all  tracks.9  Through  experimentation,  we  have  discovered  cases  in  which  an 
approach  based  purely  on  track  identification  can  lead  to  undesirable  results.  Consider  the  case  in  which  several 
targets  of  interest  (TOIs)  have  been  classified  with  a  high  degree  of  certainty.  This  scenario  is  quite  common 
as  an  EO  sensor  has  the  ability  to  offer  excellent  identification  performance.  In  this  scenario,  the  schedule 
evaluator  will  not  retask  a  sensor  on  these  TOIs  unless  they  become  confused  with  another  track  resulting  is 
loss  of  classification.  However,  such  events  may  take  several  minutes  to  occur.  Thus,  because  track  location 
uncertainty  can  increase  quickly,  leading  to  a  lost  TOI,  our  motivation  is  to  retask  on  a  well  classified  target  if 
we  have  degraded  knowledge  of  its  kinematic  state.  Approaches  to  combined  information  and  kinematic  tasking 
have  been  proposed10;  here,  we  seek  a  computationally  tractable  solution. 

The  general  idea  is  to  task  sensors  to  targets  that  have  a  large  position  covariance.  The  reasoning  is  that  the 
next  sensor  revisit  to  acquire  the  target  may  be  unsuccessful  since  knowledge  of  the  target’s  position  may  be  so 
poor  that  the  target  will  not  be  inside  the  sensor  footprint.  By  keeping  the  covariance  small  through  frequent 
updates  of  the  target’s  position,  we  are  better  able  to  keep  it  in  track.  The  location  of  the  algorithm  within  the 
overall  sensor  tasking  architecture  is  shown  in  Figure  5.  As  seen  in  the  figure,  track  loss  probability  (TLP)  is 
used  in  evaluating  the  score  of  a  schedule.  Some  of  the  difficulties  with  incorporating  TLP  into  our  cost  function 
include: 

1.  Our  tracker  extrapolates  the  track  to  multiple  states  (a  road  split  is  one  example).  Some  of  these  will 
have  a  higher  probability  of  being  the  target  location  than  others.  A  problem  arises  when  states  with  low 
probability  are  tasked  due  to  their  high  levels  of  kinematic  uncertainty. 

2.  While  the  cost  typically  weights  more  heavily  towards  TOIs  than  civilians,  our  new  cost  may  attempt  to 
go  after  civilian  targets  with  poor  position  information. 

3.  The  cost  function  currently  weights  acquiring  ambiguous  tracks  more  heavily  through  an  averaging  of 
entropies.  With  our  modified  cost,  we  could  potentially  weight  tracks  with  poor  position  knowledge  over 
ambiguous  tracks. 

As  will  be  seen  in  Section  4,  these  issues  limit  the  performance  of  schedule  evaluation  with  TLP.  The  inclusion 
of  other  techniques,  such  as  DRS,  can  overcome  certain  deficiencies. 

Before  we  present  the  modified  expression  for  entropy,  we  define  the  following  terms: 

•  A:  Represents  acquiring  the  target.  That  is,  if  we  put  down  a  footprint  the  target  is  recognized  by  the 
sensor.  Typically,  this  will  involve  the  target  being  both  inside  the  footprint  and  being  detected  by  the 
sensor.  However,  here  we  leave  the  term  general. 


Figure  5.  Role  of  Kinematics  within  sensor  tasking  architecture. 


•  L:  This  term  refers  to  target  loss  at  the  end  of  the  evaluation  time  horizon  for  sensor  schedule  evaluation. 
That  is,  if  we  were  to  revisit  the  target  with  a  standard  size  footprint,  would  we  acquire  the  target?  If  not, 
then  we  say  that  it  is  likely  the  target  will  not  be  detected  again  and  hence  the  track  will  be  lost. 


Using  total  probability,  we  expand  the  entropy  as 


H  =  ( H\A)P(A )  +  (H\A)P(A)  (8) 

where  P(A)  can  also  be  written  as  1  —  P(A).  The  conditional  entropy  expressions  can  be  expanded  as 

(H\A)  =  (H\A,L)P(L\A)  +  (H\A,L)P(L\A) 

=  HP(L\A)  +  HMaxP(L\A)  (9) 

and 

(H\A)  =  (H\A,  L)P(L\A)  +  (H\A,  L)P(L\A) 

=  HoldP(L\A)  +  HMaKP(L\A)  (10) 


where  we  have  let  H  denote  the  updated  entropy  assuming  that  the  target  is  acquired  (and  hence  gets  an  update 
via  a  sensor  measurement)  and  that  it  will  not  be  lost  at  the  end  of  the  task  evaluation  time.  For  our  second 
equation,  the  term  Horn  denotes  the  case  in  which  the  target  classification  has  not  been  updated  via  a  sensor 
measurement,  but  will  not  be  lost  due  to  poor  position  knowledge.  For  the  last  entropy  term,  if  the  target  is 
lost  then  we  assume  that  the  entropy  will  be  maximized  providing  a  high  cost  for  losing  a  target.  This  value  is 
given  by 

?W  =  log(A0  (11) 

where  we  have  assumed  that  in  the  maximum  entropy  case  all  N  types  of  targets  are  equally  likely.  Finally,  we 
note  that  we  can  make  the  approximation 

(H\A)~H  (12) 

if  we  assume  that  the  probability  of  losing  a  track  is  approximately  zero  if  the  track  was  acquired.  That  is, 
through  a  sensor  measurement  we  are  able  to  provide  sufficient  accuracy  in  the  position  measurement  such  that 
the  target  is  not  lost. 

Now  that  we  have  discussed  the  entropy  expressions,  we  can  look  at  the  expressions  for  the  probability  of 
losing  a  track  conditioned  on  the  event  that  the  target  was  not  acquired.  This  term  can  be  expanded  as 
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Figure  6.  Snapshot  of  SLAMEM  testbed  running  simulated  case. 


where  Si  denotes  extrapolated  state  l  of  the  L  states  for  the  current  target  under  consideration.  The  probabilities 
of  these  states  (P(s;))  are  given  by  the  tracker  along  with  their  associated  covariance  matrices.  To  estimate  the 
probability  of  loss,  we  will  compute  the  integral  of  the  density  over  a  circle  of  radius  7.  Letting  er2  =  y/det(C’/) 
be  an  estimate  of  the  position  covariance,  where  Ci  is  the  covariance  for  state  l,  we  can  obtain  an  expression  for 
the  density  of  the  state  position: 
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Note  that  we  have  constrained  the  density  such  that  the  x  and  y  positions  have  equal  covariance  and  are 
uncorrelated  to  ease  in  computation  of  the  integral  to  follow.  Using  a  change  of  variables  into  polar  coordinates, 
an  expression  for  the  probability  of  loss  can  be  written  as 
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Note  the  footprint  size  7  can  be  chosen  to  meet  different  goals.  Here,  we  use  the  maximum  candidate  EO  sensor 
footprint  size  in  order  to  avoid  track  loss  upon  EO  retasking.  An  alternative  would  be  to  specify  a  fixed  size  for 
cases  such  as  targeting. 


4.  COMPUTER  SIMULATIONS 

For  analysis  of  our  enhancements  to  sensor  task  creation  and  schedule  evaluation,  we  used  the  Toyon-developed 
simulation  testbed  SLAMEM™  (Simulation  of  the  Location  and  Attack  of  Mobile  Enemy  Missiles).  SLAMEM 
makes  an  excellent  testbed  for  the  development  and  testing  of  ISR  algorithms  as  it  contains  models  of  the 
relevant  components  of  the  ISR  environment  such  as  terrain  elevation  data,  road  networks,  and  foliage.  Figure  6 
provides  a  screenshot  of  the  simulation  where  vehicles,  the  road  network,  as  well  as  stand-off  aircraft  and  their 
associated  field  of  views  (FOVs)  are  shown.  SLAMEM  contains  a  detailed  ground-vehicle-motion  model  that 
takes  into  consideration  vehicle  interactions,  right-of-way  logic  at  intersections,  and  variable  vehicle  speeds. 
Sensor  platforms  such  as  satellites,  aircraft,  and  ground  vehicles  are  modeled.  Sensors  such  as  GMTI,  SAR,  and 
EO  are  modeled  to  the  level  of  detail  necessary  to  account  for  an  accurate  representation  of  where  and  when 
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Figure  7.  Monte  Carlo  simulation  with  different  combinations  of  tasking  enhancements. 


they  image  a  portion  of  the  ground.  Models  and  algorithms  for  tasking,  fusion,  and  exploitation  are  contained 
within  SLAMEM.  In  particular,  we  use  an  interacting  multiple  model  (IMM)  tracker.11 

For  our  simulation,  we  considered  an  environment  with  six  TOIs  and  40  civilian  confusers.  There  are  four 
Predator  UAVs  statically  tasked  to  fly  in  a  triangular  pattern,  where  they  are  evenly  separated  in  distance.  In 
addition,  two  GlobalHawks  provide  stand-off  radar  coverage  with  both  high-resolution  radar  (HRR)  as  well  as 
moving-target  identification  (MTI)  radar.  The  test  case  presents  a  challenging  scenario  with  such  features  as 
road  splits,  stop-move-stop,  and  obscurations.  During  simulation,  we  considered  all  combinations  of  both  DRS 
and  TLP  being  both  on  and  off.  Ten  Monte  Carlo  runs,  each  with  a  simulated  time  of  eight  hours,  were  run 
and  the  results  for  the  four  cases  are  shown  in  Figure  7.  From  the  figure,  it  is  clear  DRS  offers  performance 
gains  in  all  cases.  The  same  cannot  be  said  with  the  inclusion  of  TLP.  With  only  TLP  added,  there  is  a  period 
of  performance  gain  and  another  period  with  worse  performance.  The  major  cause  of  this  mixed  performance 
is  that  TLP  will  weight  extrapolated  states  with  high  kinematic  uncertainty  even  though  they  may  have  low 
probability  of  being  associated  with  the  track.  However,  if  DRS  is  included,  larger  footprints  often  allow  multiple 
extrapolated  states  to  be  imaged  simultaneously  overcoming  the  deficiency.  Thus,  when  both  DRS  and  TLP  are 
used  we  see  the  highest  level  of  performance. 

5.  CONCLUSION 

The  area  of  sensor  management  is  rapidly  developing  due  to  the  increasing  demands  placed  on  the  operators  of 
airborne  assets.  In  this  paper  we  addressed  the  practical  inclusion  of  kinematic  information  in  two  specific  areas 
of  an  overall  sensor  management  architecture.  In  terms  of  sensor  task  creation,  we  provided  a  DRS  algorithm 
that  is  able  to  select  an  EO  imager  parameter,  the  resolution,  that  is  optimized  in  terms  of  keeping  a  TOI 
in  track.  Similarly,  for  schedule  evaluation,  we  proposed  a  technique  whereby  TLP  can  be  integrated  into  an 
information-theoretic  based  valuation  of  a  schedule.  Through  simulation,  on  the  Toyon-developed  SLAMEM 
testbed,  we  evaluated  the  performance  of  each  algorithm  individually  as  well  as  when  both  techniques  were 


applied  to  sensor  management.  We  demonstrated  the  ability  to  significantly  increase  the  number  of  TOIs  that 
were  able  to  be  continuously  maintained  in  track  over  the  lifetime  of  the  simulation. 
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